Display device and operating method thereof

ABSTRACT

A display device includes a short-range communication unit to receive a search request signal from a mobile terminal and receive a voice signal from the mobile terminal after receiving the search request signal; and a display unit to display a result searched based on the voice signal.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a Continuation of U.S. patent application Ser. No.16/344,609 filed on Apr. 24, 2019, which is the National Phase of PCTInternational Application No. PCT/KR2016/015042 filed on Dec. 21, 2016,which claims the priority benefit under 35 U.S.C. § 119(a) to KoreanPatent Application No. 10-2016-0153405 filed in the Republic of Korea onNov. 17, 2016, all of which are hereby expressly incorporated byreference into the present application.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to a display device and a method ofoperating the same.

Discussion of the Related Art

Recently, a digital TV service using a wired or wireless communicationnetwork has come into wide use. The digital TV service may providevarious services which cannot be provided by an existing analogbroadcast service.

For example, an Internet protocol television (IPTV) service or a smartTV service which is a digital TV service provides interactivity forenabling a user to actively select a type of a program to be viewed, aviewing time, etc. The IPTV service or the smart TV service may providevarious additional services, e.g., Internet search, home shopping,online games, etc. based on such interactivity.

In addition, a display device may also provide an Internet searchservice. In particular, an Internet search, which was previouslyprovided on a computer, becomes possible on a display device. However,the display device has a disadvantage in that the manipulation such asinputting a search word or the like is not easy. In particular, when auser tries to search specific information while talking with a mobileterminal, it is difficult for the user to manipulate the manipulation.

Meanwhile, the display device is connected with a mobile terminal totransmit or receive data. The display device and the mobile terminal areconnected with each other through a wired scheme or a wireless scheme totransmit or receive various signals. In particular, since powerconsumption is rapidly reduced through Bluetooth Low Energy (BLE), thedisplay device and the mobile terminal may be always connected with eachother. The display device is connected with the mobile terminal toreceive a voice signal.

SUMMARY OF THE INVENTION

The present invention is to provide a display device and a method foroperating the same, capable of receiving a voice signal from a mobileterminal, searching for information on talking content, and displaying asearched result.

According to an embodiment of to the present invention, a display devicemay include a short-range communication unit to receive a search requestsignal from a mobile terminal in a call, a control unit to obtain aconnection state with the mobile terminal in response to receiving thesearch request signal, and to receive a voice signal from the mobileterminal if being in connected state with the mobile terminal, and adisplay unit to display a result searched based on the voice signal.

According to an embodiment of the present invention, the display devicemay further include a storage unit to store the result searched based onthe voice signal, and the control unit may store the searched result inthe storage unit or transmit the searched result to the mobile terminal.

According to an embodiment of the present invention, in the displaydevice, the display unit may display a sharing menu including an item tobe transmitted to a calling party and an item to be selected fromcontacts and to be transmitted, and the control unit may transmit thesearched result to a party making the call with the mobile terminal orto any one of contacts stored in the mobile terminal, based on selecteditem from the sharing menu.

According to an embodiment of the present invention, in the displaydevice, the control unit may obtain at least one search word byanalyzing the voice signal and perform a web-browser search, a mapsearch or a search on the mobile terminal by using the search word.

According to an embodiment of the present invention, in the displaydevice, the control unit may split a screen displayed on the displayunit into a plurality of areas and display a video, which is beingviewed, and the searched result on the plurality of areas that aresplit.

According to an embodiment of the present invention, in the displaydevice, the control unit may receive a user voice signal and a voicesignal of a calling party and display the user voice signal and thevoice signal of the calling party as captions, and display the searchedresult based on a selected caption when receiving a command forselecting any one of the captions.

According to an embodiment of the present invention, in the displaydevice, the control unit may automatically store the searched result inthe storage unit when the call is terminated.

According to an embodiment of the present invention, in the displaydevice, the control unit may store the searched result by matching thesearched result with a calling party.

According to an embodiment of the present invention, in the displaydevice, the search request signal may be a voice signal including apreset keyword.

According to an embodiment of the present invention, in the displaydevice, the control unit may turn on power of the display device anddisplay that a search is possible in response to receiving the searchrequest signal, when the power is turned off even through being in theconnected state with the mobile terminal.

Advantageous Effects

According to various embodiments of the present invention, there may beprovided a method allowing the user to easily search for specificinformation through the display device while talking on the phone. Indetail, the search may be performed through the voice signal, therebyovercoming the difficulty that the user inputs the search word whiletalking on the phone.

According to various embodiments of the present invention, theinformation desired by the user may be provided through various searchesusing the voice signal.

According to various embodiments of the present invention, talkingconversation may be checked through the caption while searching for thespecific information.

According to various embodiments of the present invention, the searchedresult may be stored corresponding to the calling party, so informationrelated to the calling party may be checked at a desired time point.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration of a displaydevice according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a remote control device accordingto an embodiment of the present invention.

FIG. 3 is a view illustrating an actual configuration of a remotecontrol device according to an embodiment of the present invention.

FIG. 4 is a view of utilizing a remote control device according to anembodiment of the present invention.

FIG. 5 is a block diagram illustrating a mobile terminal according to anembodiment of the present invention.

FIG. 6 a ladder diagram illustrating the operating method between thedisplay device and the mobile terminal according to an embodiment of thepresent invention.

FIG. 7 illustrates a conventional problem.

FIG. 8 is a view illustrating a method of receiving the search requestsignal according to a first embodiment of the present invention.

FIG. 9 is a view illustrating a method of receiving the search requestsignal according to a second embodiment of the present invention.

FIGS. 10 and 11 illustrate a method that the display device according toan embodiment of the present invention is BLE connected with the mobileterminal.

FIG. 12 is a view illustrating a method that the display deviceaccording to an embodiment of the present invention receives the powercontrol command from the mobile terminal.

FIG. 13 is a view illustrating a method that the display deviceaccording to the first embodiment of the present invention displays thescreen corresponding to the search mode.

FIG. 14 is a view illustrating a method that the display deviceaccording to the second embodiment of the present invention displays thescreen corresponding to the search mode.

FIG. 15 is a view illustrating a method that the mobile terminalaccording to an embodiment of the present invention recognizes andtransmits the voice signal.

FIGS. 16 to 22 are views illustrating a method that the display deviceaccording to the first embodiment of the present invention receives avoice signal and displays the searched result.

FIGS. 23 and 24 are views illustrating a method that the display deviceaccording to the second embodiment of the present invention receives avoice signal and displays the searched result.

FIGS. 25 and 26 are views illustrating a method that the display deviceaccording to the third embodiment of the present invention receives avoice signal and displays the searched result.

FIGS. 27 to 29 are views illustrating a method that the display deviceaccording to the fourth embodiment of the present invention receives avoice signal and displays the searched result.

FIGS. 30 to 32 are views illustrating a method that the display devicestores and displays the searched result after terminating the callaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments relating to the present invention will bedescribed in detail with reference to the accompanying drawings. Thesuffixes “module” and “unit” for components used in the descriptionbelow are assigned or mixed in consideration of easiness in writing thespecification and do not have distinctive meanings or roles bythemselves.

The display device according to an embodiment of the present invention,which is an intelligent display device that adds a computer aid functionto a broadcast receiving function, for example, and is equipped with amore convenient interface for, for example, a handwriting input device,a touchscreen, or a space remote control. In addition, the displaydevice is connected with the Internet and a computer as a wired orwireless Internet function is supported. Accordingly, a function, suchas an e-mail, a web browsing function, a banking function, or a gameplaying function, may be performed. For various functions, astandardized general-purpose OS may be used.

Therefore, the display device described in the present invention mayperform various user-friendly functions since various applications arefreely added or deleted on a general-purpose OS kernel. In more detail,the display device may include a network TV, an HBBTV, a smart TV, anLED TV, an OLED TV, or the like. Accordingly, the display device isapplicable even to a smartphone if necessary.

FIG. 1 is a block diagram illustrating a configuration of a displaydevice according to an embodiment of the present invention.

Referring to FIG. 1, a display device 100 can include a broadcastreception unit 130, an external device interface unit 135, a storageunit 140, a user input interface unit 150, a control unit 170, awireless communication unit 173, a display unit 180, an audio outputunit 185, and a power supply unit 590.

The broadcast reception unit 130 can include a tuner 131, a demodulationunit 132, and a network interface unit 133.

The tuner 131 can select a specific broadcast channel according to achannel selection command. The tuner 131 can receive broadcast signalsfor the selected specific broadcast channel.

The demodulation unit 132 can divide the received broadcast signals intovideo signals, audio signals, and broadcast program related data signalsand restore the divided video signals, audio signals, and data signalsto an output available form.

The external device interface unit 135 can receive an application or anapplication list in an adjacent external device and deliver it to thecontrol unit 170 or the storage unit 140.

The external device interface 135 can provide a connection path betweenthe display device 100 and an external device. The external deviceinterface 135 can receive at least one of image or audio outputted froman external device that is wirelessly or wiredly connected to thedisplay device 100 and deliver it to the control unit. The externaldevice interface unit 135 can include a plurality of external inputterminals. The plurality of external input terminals can include an RGBterminal, at least one High Definition Multimedia Interface (HDMI)terminal, and a component terminal.

An image signal of an external device inputted through the externaldevice interface unit 135 can be outputted through the display unit 180.A sound signal of an external device inputted through the externaldevice interface unit 135 can be outputted through the audio output unit185.

An external device connectable to the external device interface unit 130can be one of a set-top box, a Blu-ray player, a DVD player, a gameconsole, a sound bar, a smartphone, a PC, a USB Memory, and a hometheater system but this is just exemplary.

The network interface unit 133 can provide an interface for connectingthe display device 100 to a wired/wireless network including internetnetwork. The network interface unit 133 can transmit or receive data toor from another user or another electronic device through an accessednetwork or another network linked to the accessed network.

Additionally, some content data stored in the display device 100 can betransmitted to a user or an electronic device, which is selected fromother users or other electronic devices pre-registered in the displaydevice 100.

The network interface unit 133 can access a predetermined webpagethrough an accessed network or another network linked to the accessednetwork. That is, it can transmit or receive data to or from acorresponding server by accessing a predetermined webpage throughnetwork.

Then, the network interface unit 133 can receive contents or dataprovided from a content provider or a network operator. That is, thenetwork interface unit 133 can receive contents such as movies,advertisements, games, VODs, and broadcast signals, which are providedfrom a content provider or a network provider, through network andinformation relating thereto.

Additionally, the network interface unit 133 can receive firmware updateinformation and update files provided from a network operator andtransmit data to an internet or content provider or a network operator.

The network interface unit 133 can select and receive a desiredapplication among applications open to the air, through network.

The storage unit 140 can store signal-processed image, voice, or datasignals stored by a program in order for each signal processing andcontrol in the control unit 170.

Additionally, the storage unit 140 can perform a function fortemporarily store image, voice, or data signals outputted from theexternal device interface unit 135 or the network interface unit 133 andcan store information on a predetermined image through a channel memoryfunction.

The storage unit 140 can store an application or an application listinputted from the external device interface unit 135 or the networkinterface unit 133.

The display device 100 can play content files (for example, video files,still image files, music files, document files, application files, andso on) stored in the storage unit 140 and provide them to a user.

The user input interface unit 150 can deliver signals inputted from auser to the control unit 170 or deliver signals from the control unit170 to a user. For example, the user input interface unit 150 canreceive or process control signals such as power on/off, channelselection, and screen setting from the remote control device 200 ortransmit control signals from the control unit 170 to the remote controldevice 200 according to various communication methods such as Bluetooth,Bluetooth Low Energy (BLE), Ultra Wideband (WB), ZigBee, Radio Frequency(RF), and IR.

Additionally, the user input interface unit 150 can deliver, to thecontrol unit 170, control signals inputted from local keys (not shown)such as a power key, a channel key, a volume key, and a setting key.

Image signals that are image-processed in the control unit 170 can beinputted to the display unit 180 and displayed as an image correspondingto corresponding image signals. Additionally, image signals that areimage-processed in the control unit 170 can be inputted to an externaloutput device through the external device interface unit 135.

Voice signals processed in the control unit 170 can be outputted to theaudio output unit 185. Additionally, voice signals processed in thecontrol unit 170 can be inputted to an external output device throughthe external device interface unit 135.

Besides that, the control module 170 can control overall operations inthe display device 100.

Additionally, the control unit 170 can control the display device 100 bya user command or internal program inputted through the user inputinterface unit 150 and download a desired application or applicationlist into the display device 100 in access to network.

The control unit 170 can output channel information selected by a usertogether with processed image or voice signals through the display unit180 or the audio output unit 185.

Additionally, according to an external device image playback commandreceived through the user input interface unit 150, the control unit 170can output image signals or voice signals of an external device such asa camera or a camcorder, which are inputted through the external deviceinterface unit 135, through the display unit 180 or the audio outputunit 185.

Moreover, the control unit 170 can control the display unit 180 todisplay images and control broadcast images inputted through the tuner131, external input images inputted through the external deviceinterface unit 135, images inputted through the network interface unit,or images stored in the storage unit 140 to be displayed on the displayunit 180. In this case, an image displayed on the display unit 180 canbe a still image or video and also can be a 2D image or a 3D image.

Additionally, the control unit 170 can play content stored in thedisplay device 100, received broadcast content, and external inputcontent inputted from the outside, and the content can be in variousformats such as broadcast images, external input images, audio files,still images, accessed web screens, and document files.

Moreover, the wireless communication unit 173 can perform a wired orwireless communication with an external electronic device. The wirelesscommunication unit 173 can perform short-range communication with anexternal device. For this, the wireless communication unit 173 cansupport short-range communication by using at least one of Bluetooth™,Bluetooth Low Energy (BLE), Radio Frequency Identification (RFID),Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, NearField Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, orWireless Universal Serial Bus (USB) technologies. The wirelesscommunication unit 173 can support wireless communication between thedisplay device 100 and a wireless communication system, between thedisplay device 100 and another display device 100, or between networksincluding the display device 100 and another display device 100 (or anexternal server) through wireless area networks. The wireless areanetworks can be wireless personal area networks.

Herein, the other display device 100 can be a mobile terminal such as awearable device (for example, a smart watch, a smart glass, and a headmounted display (HMD)) or a smartphone, which is capable of exchangingdata (or inter-working) with the display device 100. The wirelesscommunication unit 173 can detect (or recognize) a communicable wearabledevice around the display device 100. Furthermore, if the detectedwearable device is a device authenticated to communicate with thedisplay device 100, the control unit 170 can transmit at least part ofdata processed in the display device 100 to the wearable device throughthe wireless communication unit 173. Accordingly, a user of the wearabledevice can use the data processed in the display device 100 through thewearable device.

The display unit 180 can convert image signals, data signals, or OSDsignals, which are processed in the control unit 170, or images signalsor data signals, which are received in the external device interfaceunit 135, into R, G, and B signals to generate driving signals.

Furthermore, the display device 100 shown in FIG. 1 is just oneembodiment of the present invention and thus, some of the componentsshown can be integrated, added, or omitted according to thespecification of the actually implemented display device 100.

That is, if necessary, two or more components can be integrated into onecomponent or one component can be divided into two or more componentsand configured. Additionally, a function performed by each block is todescribe an embodiment of the present invention and its specificoperation or device does not limit the scope of the present invention.

According to another embodiment of the present invention, unlike FIG. 1,the display device 100 can receive images through the network interfaceunit 133 or the external device interface unit 135 and play them withoutincluding the tuner 131 and the demodulation unit 132.

For example, the display device 100 can be divided into an imageprocessing device such as a set-top box for receiving broadcast signalsor contents according to various network services and a content playbackdevice for playing contents inputted from the image processing device.

In this case, an operating method of a display device according to anembodiment of the present invention described below can be performed byone of the display device described with reference to FIG. 1, an imageprocessing device such as the separated set-top box, and a contentplayback device including the display unit 180 and the audio output unit185.

Then, referring to FIGS. 2 and 3, a remote control device is describedaccording to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating a remote control device accordingto an embodiment of the present invention and FIG. 3 is a viewillustrating an actual configuration of a remote control deviceaccording to an embodiment of the present invention.

First, referring to FIG. 2, a remote control device 200 can include afingerprint recognition unit 210, a wireless communication unit 220, auser input unit 230, a sensor unit 240, an output unit 250, a powersupply unit 260, a storage unit 270, a control unit 280, and a voiceacquisition unit 290.

Referring to FIG. 2, the wireless communication unit 225transmits/receives signals to/from an arbitrary any one of displaydevices according to the above-mentioned embodiments of the presentinvention.

The remote control device 200 can include an RF module 221 fortransmitting/receiving signals to/from the display device 100 accordingto the RF communication standards and an IR module 223 fortransmitting/receiving signals to/from the display device 100 accordingto the IR communication standards. Additionally, the remote controldevice 200 can include a Bluetooth module 225 for transmitting/receivingsignals to/from the display device 100 according to the Bluetoothcommunication standards. Additionally, the remote control device 200 caninclude an NFC module 227 for transmitting/receiving signals to/from thedisplay device 100 according to the Near Field Communication (NFC)communication standards and a WLAN module 229 for transmitting/receivingsignals to/from the display device 100 according to the Wireless LAN(WLAN) communication standards

Additionally, the remote control device 200 can transmit signalscontaining information on a movement of the remote control device 200 tothe display device 100 through the wireless communication unit 220.

Moreover, the remote control device 200 can receive signals transmittedfrom the display device 100 through the RF module 221 and if necessary,can transmit a command on power on/off, channel change, and volumechange to the display device 100 through the IR module 223.

The user input unit 230 can be configured with a keypad button, a touchpad, or a touch screen. A user can manipulate the user input unit 230 toinput a command relating to the display device 100 to the remote controldevice 200. If the user input unit 230 includes a hard key button, auser can input a command relating to the display device 100 to theremote control device 200 through the push operation of the hard keybutton. This will be described with reference to FIG. 3.

Referring to FIG. 3, the remote control device 200 can include aplurality of buttons. The plurality of buttons can include a fingerprintrecognition button 212, a power button 231, a home button 232, a livebutton 233, an external input button 234, a voice adjustment button 235,a voice recognition button 236, a channel change button 237, a checkbutton 238, and a back button 239.

The fingerprint recognition button 212 can be a button for recognizing auser's fingerprint. According to an embodiment of the present invention,the fingerprint recognition button 212 can perform a push operation andreceive a push operation and a fingerprint recognition operation. Thepower button 231 can be button for turning on/off the power of thedisplay device 100. The power button 232 can be button for moving to thehome screen of the display device 100. The live button 233 can be abutton for displaying live broadcast programs. The external input button234 can be button for receiving an external input connected to thedisplay device 100. The voice adjustment button 235 can be button foradjusting the size of a volume outputted from the display device 100.The voice recognition button 236 can be a button for receiving user'svoice and recognizing the received voice. The channel change button 237can be a button for receiving broadcast signals of a specific broadcastchannel. The check button 238 can be a button for selecting a specificfunction and the back button 239 can be a button for returning to aprevious screen.

Again, FIG. 2 is described.

If the user input unit 230 includes a touch screen, a user can touch asoft key of the touch screen to input a command relating to the displaydevice 100 to the remote control device 200. Additionally, the userinput unit 230 can include various kinds of input means manipulated by auser, for example, a scroll key and a jog key, and this embodiment doesnot limit the scope of the present invention.

The sensor unit 240 can include a gyro sensor 241 or an accelerationsensor 243 and the gyro sensor 241 can sense information on a movementof the remote control device 200.

For example, the gyro sensor 241 can sense information on an operationof the remote control device 200 on the basis of x, y, and z axes andthe acceleration sensor 243 can sense information on a movement speed ofthe remote control device 200. Moreover, the remote control device 200can further include a distance measurement sensor and sense a distancewith respect to the display unit 180 of the display device 100.

The output unit 250 can output image or voice signals corresponding to amanipulation of the user input unit 235 or corresponding to signalstransmitted from the display device 100. A user can recognize whetherthe user input unit 235 is manipulated or the display device 100 iscontrolled through the output unit 250.

For example, the output unit 250 can include an LED module 251 forflashing, a vibration module 253 for generating vibration, a soundoutput module 255 for outputting sound, or a display module 257 foroutputting an image, if the user input unit 235 is manipulated orsignals are transmitted/received to/from the display device 100 throughthe wireless communication unit 225.

Additionally, the power supply unit 260 supplies power to the remotecontrol device 200 and if the remote control device 200 does not movefor a predetermined time, stops the power supply, so that power wastecan be reduced. The power supply unit 260 can resume the power supply ifa predetermined key provided at the remote control device 200 ismanipulated.

The storage unit 270 can store various kinds of programs and applicationdata necessary for a control or operation of the remote control device200. If the remote control device 200 transmits/receives signalswirelessly through the display device 100 and the RF module 221, theremote control device 200 and the display device 100 transmits/receivessignals through a predetermined frequency band.

The control unit 280 of the remote control device 200 can store, in thestorage unit 270, information on a frequency band fortransmitting/receiving signals to/from the display device 100 pairedwith the remote control device 200 and refer to it.

The control unit 280 controls general matters relating to a control ofthe remote control device 200. The control unit 280 can transmit asignal corresponding to a predetermined key manipulation of the userinput unit 235 or a signal corresponding to a movement of the remotecontrol device 200 sensed by the sensor unit 240 to the display device100 through the wireless communication unit 225.

Additionally, the voice acquisition unit 290 of the remote controldevice 200 can obtain voice.

The voice acquisition unit 290 can include at least one microphone 291and obtain voice through the microphone 291.

Then, FIG. 4 is described.

FIG. 4 is a view of utilizing a remote control device according to anembodiment of the present invention.

FIG. 4(a) illustrates that a pointer 205 corresponding to the remotecontrol device 200 is displayed on the display unit 180.

A user can move or rotate the remote control device 200 vertically orhorizontally. The pointer 205 displayed on the display unit 180 of thedisplay device 100 corresponds to a movement of the remote controldevice 200. Since the corresponding pointer 205 is moved and displayedaccording to a movement on a 3D space as show in the drawing, the remotecontrol device 200 can be referred to as a spatial remote controller.

FIG. 4(b) illustrates that if a user moves the remote control device200, the pointer 205 displayed on the display unit 180 of the displaydevice 100 is moved to the left in correspondence thereto.

Information on a movement of the remote control device 200 detectedthrough a sensor of the remote control device 200 is transmitted to thedisplay device 100. The display device 100 can calculate the coordinatesof the pointer 205 from the information on the movement of the remotecontrol device 200. The display device 100 can display the pointer 205to match the calculated coordinates.

FIG. 4(c) illustrates that while a specific button in the remote controldevice 200 is pressed, a user moves the remote control device 200 awayfrom the display unit 180. Thus, a selection area in the display unit180 corresponding to the pointer 205 can be zoomed in and displayedlargely.

On the other hand, if a user moves the remote control device 200 closeto the display unit 180, a selection area in the display unit 180corresponding to the pointer 205 can be zoomed out and displayedreduced.

On the other hand, if the remote control device 200 is away from thedisplay unit 180, a selection area can be zoomed out and if the remotecontrol device 200 is close to the display unit 180, a selection areacan be zoomed in.

Additionally, if a specific button in the remote control device 200 ispressed, the recognition of a vertical or horizontal movement can beexcluded. That is, if the remote control device 200 is moved away fromor close to the display unit 180, the up, down, left, or right movementcannot be recognized and only the back and forth movement can berecognized. While a specific button in the remote control device 200 isnot pressed, only the pointer 205 is moved according to the up, down,left or right movement of the remote control device 200.

Moreover, the moving speed or moving direction of the pointer 205 cancorrespond to the moving speed or moving direction of the remote controldevice 200.

Furthermore, a pointer in this specification means an object displayedon the display unit 180 in correspondence to an operation of the remotecontrol device 200. Accordingly, besides an arrow form displayed as thepointer 205 in the drawing, various forms of objects are possible. Forexample, the above concept includes a point, a cursor, a prompt, and athick outline. Then, the pointer 205 can be displayed in correspondenceto one point of a horizontal axis and a vertical axis on the displayunit 180 and also can be displayed in correspondence to a plurality ofpoints such as a line and a surface.

Then, a configuration of a mobile terminal according to an embodiment ofthe present invention will be described with reference to FIG. 5.

FIG. 5 is a block diagram illustrating a mobile terminal according to anembodiment of the present invention.

The mobile terminal 500 can include a wireless communication unit 510,an input unit 520, the sensing unit 540, an output unit 550, aninterface unit 560, the memory 570, a control unit 580, and a powersupply unit 590. In implementing a mobile terminal, components shown inFIG. 5 are not necessary, so that a mobile terminal described in thisspecification can include components less or more than the componentslisted above.

In more detail, the wireless communication unit 510 in the componentscan include at least one module allowing wireless communication betweenthe mobile terminal 500 and a wireless communication system, between themobile terminal 500 and another mobile terminal 500, or between themobile terminal 500 and an external server. Additionally, the wirelesscommunication unit 510 can include at least one module connecting themobile terminal 500 to at least one network.

The wireless communication unit 510 can include at least one of abroadcast receiving module 511, a mobile communication module 512, awireless internet module 513, a short-range communication module 514, ora location information module 515.

The input unit 520 can include a camera 521 or an image input unit forimage signal input, a microphone 522 or an audio input unit forreceiving audio signal input, and a user input unit 523 (for example, atouch key and a mechanical key)) for receiving information from a user.Voice data or image data collected by the input unit 520 are analyzedand processed as a user's control command.

The sensing unit 540 can include at least one sensor for sensing atleast one of information in a mobile terminal, environmental informationaround a mobile terminal, or user information. For example, the sensingunit 540 can include at least one of a proximity sensor 541, anillumination sensor 542, a touch sensor, an acceleration sensor, amagnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGBsensor, an infrared (IR) sensor, a finger scan sensor, an ultrasonicsensor, an optical sensor (for example, the camera 121), a microphone(for example, the microphone 122), a battery gauge, an environmentalsensor (for example, a barometer, a hygrometer, a thermometer, aradiation sensor, a thermal sensor, and a gas sensor), or a chemicalsensor (for example, an electronic noise, a healthcare sensor, and abiometric sensor). Moreover, a mobile terminal disclosed in thisspecification can combines information sensed by at least two or moresensors among such sensors and can then utilize it.

The output unit 550 is used to generate a visual, auditory, or hapticoutput and can include at least one of a display unit 551, a soundoutput unit 552, a haptic module 553, or an optical output unit 554. Thedisplay unit 551 can be formed with a mutual layer structure with atouch sensor or formed integrally, so that a touch screen can beimplemented. Such a touch screen can serve as the user input unit 523providing an input interface between the mobile terminal 500 and a userand an output interface between the mobile terminal 500 and a user atthe same time.

The interface unit 560 can serve as a path to various kinds of externaldevices connected to the mobile terminal 500. The interface unit 560 caninclude at least one of a wired/wireless headset port, an externalcharger port, a wired/wireless data port, a memory card port, a portconnecting a device equipped with an identification module, an audioInput/Output (I/O) port, a video I/O port, or an earphone port. Incorrespondence to that an external device is connected to the interfaceunit 560, the mobile terminal 500 can perform an appropriate controlrelating to the connected external device.

Additionally, the memory 570 can store data supporting various functionsof the mobile terminal 500. The memory 570 can store a plurality ofapplication programs (for example, application programs or applications)running on the mobile terminal 500 and also data and commands foroperations of the mobile terminal 500. At least part of such anapplication program can be downloaded from an external server through awireless communication. Additionally, at least part of such anapplication program can be included in the mobile terminal 500 from thetime of shipment in order to perform a basic function (for example, anincoming call, a transmission function, and a message reception) of themobile terminal 500. Moreover, an application program can be stored inthe memory 570 and installed on the mobile terminal 500, so that it canrun to perform an operation (or a function) of the mobile terminal 100by the control unit 580.

The control unit 580 can control overall operations of the mobileterminal 500 generally besides an operation relating to the applicationprogram. The control unit 580 can provide appropriate information orfunctions to a user or process them by processing signals, data, andinformation inputted/outputted through the above components or executingapplication programs stored in the memory 570.

Additionally, in order to execute an application program stored in thememory 570, the control unit 580 can control at least part of thecomponents shown in FIG. 5. Furthermore, in order to execute theapplication program, the control unit 580 can combine at least two ofthe components in the mobile terminal 500 and can then operate it.

The power supply unit 590 can receive external power or internal powerunder a control of the control unit 580 and can then supply power toeach component in the mobile terminal 500. The power supply unit 590includes a battery and the battery can be a built-in battery or areplaceable battery.

At least part of the each component can operate cooperatively in orderto implement operations, controls, or control methods of a mobileterminal 100 according to various embodiments of the present inventiondescribed below. Additionally, the operations, controls, or controlmethods of a mobile terminal 100 can be implemented on the mobileterminal 100 by executing at least one application program stored in thememory 570.

Hereinafter, prior to examining various embodiments implemented throughthe mobile terminal 500, the above-listed components are described inmore detail with reference to FIG. 5.

First, in describing the wireless communication unit 510, the broadcastreceiving module 510 of the wireless communication unit 511 can receivea broadcast signal and/or broadcast related information from an externalbroadcast management server through a broadcast channel. The broadcastchannel can include a satellite channel and a terrestrial channel. Atleast two broadcast receiving modules for simultaneous broadcastreception for at least two broadcast channels or broadcast channelswitching can be provided to the mobile terminal 100.

The mobile communication module 512 can transmit/receive a wirelesssignal to/from at least one of a base station, an external terminal, anda server on a mobile communication network established according to thetechnical standards or communication methods for mobile communication(for example, Global System for Mobile communication (GSM), CodeDivision Multi Access (CDMA), Code Division Multi Access 2000(CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only(EV-DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access(HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution(LTE), or Long Term Evolution-Advanced (LTE-A)).

The wireless signal can include various types of data according to avoice call signal, a video call signal, or text/multimedia messagetransmission.

The wireless internet module 513 refers to a module for wirelessinternet access and can be built in or external to the mobile terminal500. The wireless internet module 513 can be configured totransmit/receive a wireless signal in a communication network accordingto wireless internet technologies.

The wireless internet technology can include Wireless LAN (WLAN),Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance(DLNA), Wireless Broadband (WiBro), World Interoperability for MicrowaveAccess (WiMAX), High Speed Downlink Packet Access (HSDPA), High SpeedUplink Packet Access (HSUPA), Long Term Evolution (LTE), and Long TermEvolution-Advanced (LTE-A) and the wireless internet module 513transmits/receives data according at least one wireless internettechnology including internet technology not listed above.

From the viewpoint that wireless internet access by WiBro, HSDPA, HSUPA,GSM, CDMA, WCDMA, LTE, and LTE-A is achieved through a mobilecommunication network, the wireless internet module 513 performingwireless internet access through the mobile communication network can beunderstood as one type of the mobile communication module 512.

The short-range communication module 514 can support short-rangecommunication by using at least one of Bluetooth™, Bluetooth Low Energy(BLE), Radio Frequency Identification (RFID), Infrared Data Association(IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC),Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, or Wireless Universal SerialBus (USB) technologies. The short-range communication module 514 cansupport wireless communication between the mobile terminal 500 and awireless communication system, between the mobile terminal 500 andanother mobile terminal 500, or between networks including the mobileterminal 500 and another mobile terminal 100 (or an external server)through wireless area networks. The wireless area networks can bewireless personal area networks.

Here, the other mobile terminal 500 can be a wearable device (forexample, a smart watch, a smart glass, and an HMD) that is capable ofexchanging data (or interworking) with the mobile terminal 500. Theshort-range communication module 514 can detect (or recognize) awearable device around the mobile terminal 500, which is capable ofcommunicating with the mobile terminal 500 Furthermore, if the detectedwearable device is a device authenticated to communicate with the mobileterminal 500, the control unit 580 can transmit at least part of dataprocessed in the mobile terminal 500 to the wearable device through theshort-range communication module 514. Accordingly, a user of thewearable device can use the data processed in the mobile terminal 500through the wearable device. For example, according thereto, if a callis received by the mobile terminal 500, a user can perform a phone callthrough the wearable device or if a message is received by the mobileterminal 500, a user can check the received message.

The location information module 515 is a module for obtaining thelocation (or the current location) of a mobile terminal and itsrepresentative examples include a global positioning system (GPS) moduleor a Wi-Fi module. For example, the mobile terminal can obtain itsposition by using a signal transmitted from a GPS satellite through theGPS module. As another example, the mobile terminal can obtain itsposition on the basis of information of a wireless access point (AP)transmitting/receiving a wireless signal to/from the Wi-Fi module,through the Wi-Fi module. If necessary, the position information module115 can perform a function of another module in the wirelesscommunication unit 510 in order to obtain data on the location of themobile terminal substitutionally or additionally. The locationinformation module 115 is a module for obtaining the position (or thecurrent position) of the mobile terminal and is not limited to a moduledirectly calculating and obtaining the position of the mobile terminal.

Then, the input unit 520 is used for inputting image information (orsignal), audio information (or signal), data, or information inputtedfrom a user and the mobile terminal 500 can include at least one camera521 in order for inputting image information. The camera 521 processesimage frames such as a still image or a video obtained by an imagesensor in a video call mode or a capturing mode. The processed imageframe can be displayed on the display unit 551 or stored in the memory570. Moreover, a plurality of cameras 521 equipped in the mobileterminal 500 can be arranged in a matrix structure and through thecamera 521 having such a matrix structure, a plurality of imageinformation having various angles or focuses can be inputted to theinput terminal 500. Additionally, the plurality of cameras 521 can bearranged in a stereo structure to obtain the left and right images forimplementing a three-dimensional image.

The microphone 522 processes external sound signals as electrical voicedata. The processed voice data can be utilized variously according to afunction (or an application program being executed) being performed inthe mobile terminal 500. Moreover, various noise canceling algorithmsfor removing noise occurring during the reception of external soundsignals can be implemented in the microphone 522.

The user input unit 523 is to receive information from a user and ifinformation is inputted through the user input unit 523, the controlunit 580 can control an operation of the mobile terminal 500 tocorrespond to the inputted information. The user input unit 523 caninclude a mechanical input means (or a mechanical key, for example, abutton, a dome switch, a jog wheel, and a jog switch at the front, backor side of the mobile terminal 500) and a touch type input means. As oneexample, a touch type input means can include a virtual key, a soft key,or a visual key, which is displayed on a touch screen through softwareprocessing or can include a touch key disposed at a portion other thanthe touch screen. Moreover, the virtual key or visual key can havevarious forms and can be disposed on a touch screen and for example, caninclude graphic, text, icon, video, or a combination thereof.

Moreover, the sensing unit 540 can sense at least one of information ina mobile terminal, environmental information around a mobile terminal,or user information and can then generate a sensing signal correspondingthereto. On the basis of such a sensing signal, the control unit 580 cancontrol the drive or control of the mobile terminal 500 or can performdata processing, functions, or operations relating to an applicationprogram installed in the mobile terminal 500. Representative sensorsamong various sensors included in the sensing unit 540 will be describedin more detail.

First, the proximity sensor 541 refers to a sensor detecting whetherthere is an object approaching a predetermined detection surface orwhether there is an object around by using the strength of anelectromagnetic field or infrared, without mechanical contact. Theproximity sensor 541 can disposed in an inner area of a mobile terminalsurrounded by the touch screen or around the touch screen.

Examples of the proximity sensor 541 can include a transmission-typephotoelectric sensor, a direct reflective-type photoelectric sensor, amirror reflective-type photoelectric sensor, a high-frequencyoscillation-type proximity sensor, a capacitive-type proximity sensors,a magnetic-type proximity sensor, and an infrared proximity sensor. Ifthe touch screen is a capacitive type, the proximity sensor 541 can beconfigured to detect the proximity of an object by changes in anelectric field according to the proximity of the object havingconductivity. In this case, the touch screen (or a touch sensor) itselfcan be classified as a proximity sensor.

Moreover, for convenience of description, an action for recognizing theposition of an object on the touch screen as the object is close to thetouch screen without contacting the touch screen is called “proximitytouch” and an action that the object actually contacts the touch screenis called “contact touch”. A position that an object isproximity-touched on the touch screen is a position that the objectvertically corresponds to the touch screen if the object isproximity-touched. The proximity sensor 541 can detect a proximity touchand a proximity touch pattern (for example, a proximity touch distance,a proximity touch direction, a proximity touch speed, a proximity touchtime, a proximity touch position, and a proximity touch movement state).Moreover, the control unit 580 processes data (for information)corresponding to a proximity touch operation and a proximity touchpattern, detected through the proximity sensor 541, and furthermore, canoutput visual information corresponding to the processed data on thetouch screen. Furthermore, according to whether a touch for the samepoint on the touch screen is a proximity touch or a contact touch, thecontrol unit 580 can control the mobile terminal 500 to processdifferent operations or data (or information).

The touch sensor detects a touch (or a touch input) applied to the touchscreen (or the display unit 551) by using at least one of various touchmethods, for example, a resistive film method, a capacitive method, aninfrared method, an ultrasonic method, or a magnetic field method.

For example, the touch sensor can be configured to convert a pressureapplied to a specific portion of the touch screen or changes incapacitance occurring at a specific portion into electrical inputsignals. The touch sensor can be configured to detect a position andarea that a touch target applying a touch on the touch screen touchesthe touch sensor, a pressured if touched, and a capacitance if touched.Here, the touch target, as an object applying a touch on the touchsensor, can be a finger, a touch pen, a stylus pen, or a pointer, forexample.

In such a manner, if there is a touch input on the touch sensor,signal(s) corresponding thereto are sent to a touch controller. Thetouch controller processes the signal(s) and then transmitscorresponding data to the control unit 580. Therefore, the control unit580 can recognize which area of the display unit 551 is touched. Herein,the touch controller can be an additional component separated from thecontrol unit 580 or can be the control unit 580 itself.

Moreover, the control unit 580 can perform different controls or thesame control according to types of a touch target touching the touchscreen (or a touch key equipped separated from the touch screen).Whether to perform different controls or the same control according totypes of a touch target can be determined according to a currentoperation state of the mobile terminal 500 or an application program inexecution.

Moreover, the above-mentioned touch sensor and proximity sensor areprovided separately or combined and can thus sense various types oftouches, for example, short (or tap) touch), long touch, multi touch,drag touch, flick touch, pinch-in touch, pinch-out touch, swipe touch,and hovering touch for the touch screen.

The ultrasonic sensor can recognize position information of a detectiontarget by using ultrasonic waves. Moreover, the control unit 580 cancalculate the position of a wave source through information detected byan optical sensor and a plurality of ultrasonic sensors. The position ofthe wave source can be calculated by using the property that light ismuch faster than ultrasonic wave, that is, a time that light reaches anoptical signal is much shorter than a time that ultrasonic wave reachesan ultrasonic sensor. In more detail, the position of the wave sourcecan be calculated by using a time difference with a time that ultrasonicwave reaches by using light as a reference signal.

Moreover, the camera 520 described as a configuration of the input unit521 can include at least one of a camera sensor (for example, CCD andCMOS), a photo sensor (or an image sensor), or a laser sensor.

The camera 521 and the laser sensor can be combined to detect a touch ofa detection target for a three-dimensional image. The photo sensor canbe stacked on a display device and is configured to scan a movement of adetection target close to the touch screen. In more detail, the photosensor mounts a photo diode and a transistor (TR) in a row/column andscans content disposed on the photo sensor by using an electrical signalchanging according to an amount of light applied to the photo diode.That is, the photo sensor can calculate the coordinates of a detectiontarget according to the amount of change in light and through this, canobtain the position information of the detection target.

The display unit 551 can display (output) information processed in themobile terminal 500. For example, the display unit 551 can displayexecution screen information of an application program running on themobile terminal 500 or user interface (UI) and graphic user interface(GUI) information according to such execution screen information.

Additionally, the display unit 551 can be configured as athree-dimensional display unit displaying a three-dimensional image.

A three-dimensional display method, for example, a stereoscopic method(a glasses method), an autostereoscopic (no glasses method), aprojection method (a holographic method) can be applied to thethree-dimensional display unit

The sound output unit 552 can output audio data received from thewireless communication unit 510 or stored in the memory 570 in a callsignal reception or call mode, a recording mode, a voice recognitionmode, or a broadcast reception mode. The sound output unit 552 canoutput a sound signal relating to a function (for example, a call signalreception sound and a message reception sound) performed by the mobileterminal 500. The sound output unit 552 can include a receiver, aspeaker, and a buzzer.

The haptic module 553 generates various haptic effects that a user canfeel. A representative example of a haptic effect that the haptic module553 generates is vibration. The intensity and pattern of vibrationgenerated by the haptic module 553 can be controlled by a user'sselection or a setting of a control unit. For example, the haptic module553 can synthesize and output different vibrations or output differentvibrations sequentially.

The haptic module 553 can generate various haptic effects, for example,effects by a pin arrangement moving vertical to a contact skin surface,injection power or suction power of air through an injection port or asuction port, rubbing a skin surface, electrode contact, stimulus ofelectrostatic force and effects by the reproduction of cold/warm senseby using a device absorbing or emitting heat.

The haptic module 553 can be implemented to deliver a haptic effectthrough a direct contact and also allow a user to feel a haptic effectthrough a muscle sense such as a finger or an arm. The haptic module 553can be more than two according to a configuration aspect of the mobileterminal 500.

The optical output unit 554 outputs a signal for notifying eventoccurrence by using light of a light source of the mobile terminal 500.An example of an event occurring in the mobile terminal 500 includesmessage reception, call signal reception, missed calls, alarm, schedulenotification, e-mail reception, and information reception through anapplication.

A signal outputted from the optical output unit 554 is implemented as amobile terminal emits single color of multi-color to the front or theback. The signal output can be terminated if a mobile terminal detectsuser's event confirmation.

The interface unit 560 can serve as a path to all external devicesconnected to the mobile terminal 500. The interface unit 560 can receivedata from an external device, receive power and deliver it to eachcomponent in the mobile terminal 500, or transmit data in the mobileterminal 500 to an external device. For example, the interface unit 560can include a wired/wireless headset port, an external charger port, awired/wireless data port, a memory card port, a port connecting a deviceequipped with an identification module, an audio I/O port, a video I/Oport, and an earphone port.

Moreover, the identification module, as a chip storing variousinformation for authenticating usage authority of the mobile terminal500, can include a user identity module (UIM), a subscriber identitymodule (SIM), and a universal subscriber identity module (USIM). Adevice equipped with an identification module (hereinafter referred toas an identification device) can be manufactured in a smart card form.Accordingly, the identification device can be connected to the terminal100 through the interface unit 560.

Additionally, if the mobile terminal 500 is connected to an externalcradle, the interface unit 560 can become a path through which power ofthe cradle is supplied to the mobile terminal 500 or a path throughwhich various command signals inputted from the cradle are delivered tothe mobile terminal 500 by a user. The various command signals or thepower inputted from the cradle can operate as a signal for recognizingthat the mobile terminal 500 is accurately mounted on the cradle.

The memory 570 can store a program for an operation of the control unit580 and can temporarily store input/output data (for example, a phonebook, a message, a still image, and a video). The memory 570 can storedata on various patterns of vibrations and sounds outputted during atouch input on the touch screen.

The memory 570 can include at least one type of storage medium amongflash memory type, hard disk type, Solid State Disk (SSD) type, SiliconDisk Drive (SDD) type, multimedia card micro type, card type memory (forexample, SD or XD memory type), random access memory (RAM) type, staticrandom access memory (SRAM) type, read-only memory (ROM) type,electrically erasable programmable read-only memory (EEPROM) type,programmable read-only memory (PROM) type, magnetic memory type,magnetic disk type, and optical disk type. The mobile terminal 500 canoperate in relation to a web storage performing a storage function ofthe memory 570 on internet.

Moreover, as mentioned above, the control unit 580 can controloperations relating to an application program and overall operations ofthe mobile terminal 500 in general. For example, if a state of themobile terminal 100 satisfies set conditions, the control unit 580 canexecute or release a lock state limiting an output of a control commandof a user for applications.

Additionally, the control unit 580 can perform a control or processingrelating to a voice call, data communication, and a video call canperform pattern recognition processing for recognizing handwriting inputor drawing input on the touch screen as a text and an image,respectively. Furthermore, the control unit 580 can use at least one ora combination of the above components to perform a control in order toimplement various embodiments described below on the mobile terminal500.

The power supply unit 590 can receive external power or internal powerunder a control of the control unit 580 and can then supply powernecessary for an operation of each component. The power supply unit 590includes a battery. The battery is a rechargeable built-in battery andcan be detachably coupled to a terminal body in order for charging.

Additionally, the power supply unit 590 can include a connection portand the connection port can be configured as one example of theinterface unit 160 to which an external charger supplying power forcharging of the battery is electrically connected.

As another example, the power supply unit 590 can be configured tocharge a battery through a wireless method without using the connectionport. In this case, the power supply unit 590 can receive power from anexternal wireless power transmission device through at least one of aninductive coupling method based on a magnetic induction phenomenon, or amagnetic resonance coupling method based on an electromagnetic resonancephenomenon.

Moreover, various embodiments below can be implemented in a computer ordevice similar thereto readable medium by using software, hardware, or acombination thereof.

Then, a communication system using the mobile terminal 500 is describedaccording to an embodiment of the present invention.

First, the communication system can use different wireless interfacesand/or physical layers. For example, a wireless interface available tothe communication system can include Frequency Division Multiple Access(FDMA), Time Division Multiple Access (TDMA), Code Division MultipleAccess (CDMA), and Universal Mobile Telecommunications Systems (UMTS)(especially, Long Term Evolution (LTE), Long Term Evolution-Advanced(LTE-A), and Global System for Mobile Communications (GSM)).

Hereinafter, for convenience of description, description is made limitedto CDMA. However, it is apparent that the present invention isapplicable to all communication systems including Orthogonal FrequencyDivision Multiplexing (OFDM) wireless communication systems in additionto CDMA wireless communication systems.

The CDMA wireless communication system can include at least one terminal100, at least one base station (BS) (it can be referred to as Node B orEvolved Node B), at least one base station controllers (BSCs), and amobile switching center (MSC). MSC can be configured to be connected toPublic Switched Telephone Network (PSTN) and BSCs. BSCs can be connectedbeing paired with a BS through a backhaul line. The backhaul line can beprovided according to at least one of E1/T1, ATM, IP, PPP, Frame Relay,HDSL, ADSL, or xDSL. Accordingly, a plurality of BSCs can be included ina CDMA wireless communication system.

Each of a plurality of BSs can include at least one sensor and eachsensor can include an omni-directional antenna or an antenna indicatinga specific radial direction from a BS. Additionally, each sensor caninclude at least two antennas in various forms. Each BS can beconfigured to support a plurality of frequency allocations and each ofthe plurality of frequency allocations can have a specific spectrum (forexample, 1.25 MHz, 5 MHz, and so on).

The intersection of a sector and a frequency allocation can be referredto as a CDMA channel. A BS can be referred to as a Base StationTransceiver Subsystem (BTS). In such a case, one BSC and at least one BStogether can be referred to as “BS”. ABS can also represent “cell site”.Additionally, each of a plurality of sectors for a specific BS can bereferred to as a plurality of cell sites.

A Broadcasting Transmitter (BT) transmits broadcast signals to theterminals 100 operating in a system. The broadcast reception module 511shown in FIG. 5 is provided in the terminal 100 for receiving broadcastsignals transmitted from the BT.

Additionally, GPS can be linked to a CDMA wireless communication systemin order to check the location of the mobile terminal 500. Then, asatellite helps obtaining the location of the mobile terminal 500.Useful location information can be obtained by at least one satellite.Herein, the location of the mobile terminal 500 can be traced by usingall techniques for tracing the location in addition to GPS trackingtechnique. Additionally, at least one GPS satellite can be responsiblefor satellite DMB transmission selectively or additionally.

The location information module 515 in a mobile terminal is fordetecting and calculating the position of the mobile terminal and itsrepresentative example can include a GPS module and a WiFi module. Ifnecessary, the position information module 115 can perform a function ofanother module in the wireless communication unit 510 in order to obtaindata on the location of the mobile terminal substitutionally oradditionally.

The GPS module 115 can calculate information on a distance from at leastthree satellites and accurate time information and then applytriangulation to the calculated information, in order to accuratelycalculate the 3D current location information according to latitude,longitude, and altitude. A method for calculating location and timeinformation by using three satellites and correcting errors of thecalculated location and time information by using another one satelliteis being widely used. Additionally, the GPS module 115 can speedinformation as continuously calculating the current location in realtime. However, it is difficult to accurately measure the location of amobile terminal by using a GPS module in a shadow area of a satellitesignal such as a room. Accordingly, in order to compensate for themeasurement of a GPS method, a WiFi Positioning System (WPS) can beutilized.

WPS is a technique for tracking the location of the mobile terminal 500by using a WiFi module in the mobile terminal 500 and a wireless AccessPoint (AP) for transmitting or receiving wireless signals to or from theWiFi module and can mean a Wireless Local Area Network (WLAN) basedlocation measurement technique using WiFi.

A WiFi location tracking system can include a WiFi location measurementserver, a mobile terminal 500, a wireless AP connected to the mobileterminal 500, and a database for storing arbitrary wireless APinformation.

The mobile terminal 500 in access to a wireless AP can transmit alocation information request message to a WiFi location measurementserver.

The WiFi location measurement server extracts information of a wirelessAP connected to the mobile terminal 500 on the basis of a locationinformation request message (or signal) of the mobile terminal 500.Information of a wireless AP connected to the mobile terminal 500 can betransmitted to the WiFi location measurement server through the mobileterminal 500 or can be transmitted from a wireless AP to a WiFi locationmeasurement server.

Based on the location information request message of the mobile terminal500, the extracted information of a wireless AP can be at least one ofMAC Address, Service Set Identification (SSID), Received Signal StrengthIndicator (RSSI), Reference Signal Received Power (RSRP), ReferenceSignal Received Quality (RSRQ), channel information, Privacy, NetworkType, Signal Strength, or Noise Strength.

As mentioned above, the WiFi position measurement server can extractwireless AP information corresponding to a wireless AP that the mobileterminal 500 access from a pre-established database by using informationof the wireless AP connected to the mobile terminal 100. At this point,information of arbitrary wireless APs stored in the database caninformation such as MAC Address, SSID, channel information, Privacy,Network Type, latitude and longitude coordinates of a wireless AP, abuilding name where a wireless AP is located, the number of floors,indoor detailed location (GPS coordinates available), the address of theowner of an AP, and phone numbers. At this point, in order to remove amobile AP or a wireless AP provided using illegal MAC address during ameasurement process, a WiFi location measurement server can extract onlya predetermined number of wireless AP information in high RSSI order.

Then, the WiFi location measurement server can extract (or analyze) thelocation information of the mobile terminal 500 by using at least onewireless AP information extracted from the database. By comparing theincluded information and the received wireless AP information, locationinformation of the mobile terminal 500 is extracted (or analyzed).

As a method of extracting (or analyzing) the location information of themotile terminal 500, a Cell-ID method, a finger-print method, atriangulation method, and a landmark method can be used.

The Cell-ID method is a method for determining the location of awireless AP having the strongest signal intensity in neighbor wirelessAP information that a mobile terminal collects as the location of themobile terminal. Implementation is simple, no additional cost isrequired, and location information is obtained quickly but if theinstallation density of wireless APs is low, measurement precision ispoor.

The finger-print method is a method for collecting signal intensityinformation by selecting a reference location from a service area andestimating the location through signal intensity information transmittedfrom a mobile terminal on the basis of the collected information. Inorder to use the finger-print method, there is a need to provide adatabase for storing propagation characteristics in advance.

The triangulation method is a method for calculating the location of amobile terminal on the basis of a distance between coordinates of atleast three wireless APs and a mobile terminal. In order to measure adistance between a mobile terminal and a wireless AP, a signal intensityconverted into distance information, Time of Arrival (ToA), TimeDifference of Arrival (TDoA), and Angle of Arrival (AoA) can be used.

The landmark method is a method for measuring the location of a mobileterminal by using a landmark transmitter knowing the location.

In addition to the listed methods, a variety of algorithms can beutilized as methods for extracting (or analyzing) the locationinformation of a mobile terminal.

As the extracted location information of the mobile terminal 500 istransmitted to the mobile terminal 500 through the WiFi locationmeasurement server, the mobile terminal 500 can obtain the locationinformation.

As connected to at least one wireless AP, the mobile terminal 500 canobtain location information. At this point, the number of wireless APs,which are required for obtaining the location information of the mobileterminal 500, can vary according to a wireless communication environmentwhere the mobile terminal 500 is located.

Hereinafter, a method for operating the display device 100 according toan embodiment of the present invention will be described with referenceto FIGS. 6 to 32.

First, FIG. 6 a ladder diagram illustrating the operating method betweenthe display device and the mobile terminal according to an embodiment ofthe present invention.

The wireless communication unit 510 of the mobile terminal 500 maytransmit a search request signal to the display device 100 (S101).

The search request signal according to an embodiment of the presentinvention may be a signal allowing the display device 100 to operate ina search mode. The search mode according to an embodiment of the presentinvention may be a mode in which the display device 100 may receive avoice signal from the mobile terminal 500 and may search for informationassociated with the received voice signal.

When the display device 100 operates in the search mode, a user mayeasily search for specific information while talking by the mobileterminal 500. For example, the user may wonder about the specificinformation while making a voice call. When the user carries out searchusing the mobile terminal 500, the user has to stop the talking orconversation. However, when the user carries out the search using thedisplay device 100, the user may conveniently search for what the userwonders while keeping making a voice call with the outer party. Inparticular, when the search is performed through the voice signal of theuser, it may be overcome the difficulty that the user has to input asearch word while taking by the mobile terminal 500.

Next, a method that the display device 100 according to an embodiment ofthe present invention receives the search request signal from the mobileterminal 500 will be described with reference to FIGS. 7 to 9.

FIG. 7 illustrates a conventional problem. As illustrated in FIG. 7, theuser may be talking on the mobile terminal 500. The user may want tosearch for specific information during making the voice call. The usermay search for the specific information through the display device 100adjacent to the mobile terminal 500.

FIG. 8 is a view illustrating a method of receiving the search requestsignal according to a first embodiment of the present invention. Asillustrated in FIG. 8, the control unit 170 of the display device 100may receive the search request signal as a voice signal including apreset keyword is recognized.

In detail, the control unit 580 of the mobile terminal 500 may recognizethe voice signal in a call connection state. When the voice signalincluding the preset keyword is recognized, the control unit 580 of themobile terminal 500 may transmit the recognized voice signal to thedisplay device 100. The control unit 170 of the display device 100 mayreceive the voice signal including the preset keyword. As mentionedabove, the display device 100 may receive the search request signalthrough the voice signal including the preset keyword.

The keyword may be set by default. Alternatively, the keyword may bearbitrarily set through a user input. Although FIG. 8 illustrates thatthe keyword is set to “Do you know what ˜” or “Let me know what ˜” byway of example, this is provided only for the illustrative purpose.

FIG. 9 is a view illustrating a method of receiving the search requestsignal according to a second embodiment of the present invention.

The display device 100 may include a camera 160. The camera 160 maycapture an image of an area closer to the display device 100. Thecontrol unit 170 of the display device 100 may sense the eyes of theuser through the camera 160.

In addition, the display device 100 may have a BLE connection state withat least one mobile terminal 500. The control unit 170 of the displaydevice 100 may sense that any one of mobile terminals 500 having BLEconnection state is in the call connection state.

The control unit 170 of the display device 100 may receive the searchrequest signal as the control unit 170 senses any one mobile terminal500 being in the call connection state and senses the eyes of the userfacing the direction of the display device 100 for a preset time.

In addition, the display device 100 may receive the search requestsignal through another method. For example, the control unit 580 of themobile terminal 500 may transmit the search request signal to thedisplay device 100 when analyzing the voice of the user and recognizinga questionable tone. As another example, when the mobile terminal 500 ischanged to be in the call connection state, the display device 100 mayreceive the search request signal.

As mentioned above, the control unit 170 of the display device 100 mayreceive the search request signal from the mobile terminal 500.

The following description will be made with reference to FIG. 6 again.

The control unit 170 of the display device 100 may determine a powerstate of the display device 100 and the BLE connection state with themobile terminal 500 corresponding to the search request signal (S103).

According to the present invention, the power state of the displaydevice 100 may correspond to any one of an On state and an Off state.

In addition, the display device 100 may have any one of an On state andan Off state in Bluetooth Low Energy (BLE) connection state with aspecified mobile terminal 500.

Accordingly, the display device 100 may correspond to any one of 1) apower On state and a BLE On state, 2) a power On state and a BLE Offstate, 3) a power Off state and the BLE On state, and 4) a power Offstate and a BLE Off state.

Whether the display device 100 operates depends on the power state andthe BLE connection state of the display device 100. Accordingly, thecontrol unit 170 of the display device 100 may determine both the powerstate and the BLE connection state in response to receiving the searchrequest signal.

First, the case that both the power state and the BLE connection stateare On states will be described below.

The control unit 170 may determine that both the power state and the BLEconnection state are On states. In this case, the control unit 170 maycontrol the display device 100 to be in the search mode (S119). Thedetails of the search mode will be described below.

Second, the case that the power state is an On state and the BLEconnection state is an Off state will be described below.

The control unit 170 may determine that only the power state is an Onstate (S107).

The control unit 170 may be BLE connected with the mobile terminal 500(S117).

In detail, the control unit 170 may be BLE connected with the mobileterminal 500 when the BLE connection state with the mobile terminal 500is the Off state even through the power state is an On state. Inparticular, the control unit 170 may be BLE connected with the mobileterminal 500 corresponding to the received search request signal.

FIGS. 10 and 11 illustrate a method that the display device according toan embodiment of the present invention is BLE connected with the mobileterminal.

The control unit 170 of the display device 100 may control the displayunit 180 to display a notification window 1000 on the as illustrated inFIG. 10 when only the power state is an On state. The display unit 180may display the notification window 1000 including a request message forthe BLE connection with the mobile terminal 500. For example, thedisplay unit 180 may display the notification window 1000 including amessage of “Please, connect with the mobile terminal through BLE”.

The user may confirm the notification message 1000 displayed on thedisplay unit 180 and may request for the BLE connection between thedisplay device 100 and the mobile terminal 500. For example, as shown inFIG. 11, a user can request a BLE connection between the display device100 and the mobile terminal 500 through a button provided in the remotecontrol device 200.

The control unit 170 of the display device 100 may receive the BLEconnection request through the remote control device 200 and may be BLEconnected with the mobile terminal 500. In this case, the mobileterminal 500 may be the mobile terminal 500 that having transmitted thesearch request signal.

The control unit 170 of the display device 100 may control the displayunit 180 to display a BLE connection notification window 1100 during theBLE connection with the mobile terminal 500. The BLE connectionnotification window 1100 may include messages, such as “underconnection”, “connection failure”, “connection completion” or the like.The user may determine the BLE connection state between the displaydevice 100 and the mobile terminal 500 through the BLE connectionnotification window 1100.

The following description will be made with reference to FIG. 6 again.

When the BLE connection with the mobile terminal 500 is completed(S117), the control unit 170 may control the display device 100 to be inthe search mode (S119). The details of the search mode will be describedbelow.

Third, the case in which the power supply state is an Off state and theBLE connection state is an ON state will be described below.

The control unit 170 may determine that only the BLE connection state isan On state (S109).

In this case, the mobile terminal 500 may receive an input forcontrolling power of the display device 100 (S111).

The control unit 580 of the mobile terminal 500 may transmit a powercontrol command to the display device 100 (S113).

Upon receiving the power control command from the mobile terminal 500,the control unit 170 of the display device 1000 may control the powerstate to be an On state (S115).

FIG. 12 is a view illustrating a method that the display deviceaccording to an embodiment of the present invention receives the powercontrol command from the mobile terminal.

As illustrated in FIG. 12, the display device 100 may have been alreadyconnected with the mobile terminal 500 through the BLE even though thepower of the display device 100 is an Off state. In this case, thecontrol unit 170 of the display device 100 may power on by receiving thepower control command from the mobile terminal 500. To this end, themobile terminal 500 may receive an input for controlling power of thedisplay device 100. The mobile terminal 500 may transmit the powercontrol command to the display device 100.

The input for controlling the power may be an input for selecting aspecific button provided in the mobile terminal 500. Alternatively, theinput for controlling the power may be a trigger signal. For example,the trigger signal may be a specific voice signal. As illustrated inFIG. 12, the control unit 580 of the mobile terminal 500 may receive theinput for controlling the power by sensing a preset voice signal such as“Hi TV”. Accordingly, the control unit 580 of the mobile terminal 500may transmit the power control command to the display device 100 and thedisplay device 100 may turn on the power. The control unit 170 of thedisplay device 100 may perform a control operation to operate in thesearch mode while turning on the power. In other words, the control unit170 may turn on the power while displaying a window indicating thatsearch is possible.

When the power of the display device 100 is turned off, the user mayturn on the power of the display device 100 through the mobile terminal500.

The following description will be made with reference to FIG. 6 again.

Fourthly, the case that both the power state and the BLE connectionstate are Off states will be described below.

The control unit 170 may determine that both the power state and the BLEconnection state are Off states. In this case, the control unit 170 mayattempt to make the BLE connection with the mobile terminal 500 afterturning on the power.

The control unit 170 may control the power state and the BLE connectionstate to be On states through steps S103, S105, S107, S117, S109, S111,S113, and S115 described above.

Alternatively, if the user recognizes the power state and the BLEconnection state of the display device 100, steps S103, S105, S107,S117, S109, S111, S113, and S115 may be omitted. In other words, sincethe user directly control the power state and the BLE connection stateof the display device 100, the steps for determining the power state andthe BLE connection state may be omitted.

When the control unit 170 may determine that both the power state andthe BLE connection state are On states, the control unit 170 may performa control operation such that the operation in the search mode isperformed (S119).

The search mode may be a mode in which the display device 100 mayreceive a voice signal from the mobile terminal 500 and may search forinformation associated with the received voice signal.

The control unit 170 of the display device 100 may control the displayunit 180 to display a screen corresponding to the search mode.

FIG. 13 is a view illustrating a method that the display deviceaccording to the first embodiment of the present invention displays thescreen corresponding to the search mode.

As illustrated in FIG. 13, the control unit 170 of the display device100 may control the display unit 180 to display a web search window 1300when the display device 100 operates in the search mode. In detail, thecontrol unit 170 may control the display unit 180 to display the websearch window 1300 together with a video. The web search window 1300 mayinclude a search window and a keypad. The control unit 170 may searchfor information associated with the input search word by receiving aninput of a search word through the web search window 1300.

The user may move a pointer 205 through the remote control device 100.The user may search for specific information on the web search window1300 by moving the pointer 205.

FIG. 14 is a view illustrating a method that the display deviceaccording to the second embodiment of the present invention displays thescreen corresponding to the search mode.

As illustrated in FIG. 14, the control unit 170 of the display device100 may control the display unit 180 to display a voice search waitingwindow 1400 when the display device 100 operates in the search mode.

The voice search waiting window 1400 indicates that the display device100 operates in the search mode. In other words, the voice searchwaiting window 1400 is to notify the user of that the display device 100receives a voice signal from the mobile terminal 500 and is in a statethat search is possible. The user may recognize that the user may viewthe voice search waiting window 1400 and may search for specificinformation through the mobile terminal 500

The web search window 1300 and the voice search waiting window 1400illustrated FIGS. 13 and 14 are provided for the illustrative purpose,and the present invention is not limited thereto.

The following description will be made with reference to FIG. 6 again.

The mobile terminal 500 may recognize the voice signal (S121).

In detail, the input unit 520 of the mobile terminal 500 may recognizethe voice signal.

The control unit 580 of the mobile terminal 500 may perform a controloperation to transmit the recognized voice signal to the display device100 (S123).

In detail, the control unit 580 may control the wireless communicationunit 510 to transmit the recognized voice signal to the display device100.

FIG. 15 is a view illustrating a method that the mobile terminalaccording to an embodiment of the present invention recognizes andtransmits the voice signal.

As illustrated in FIG. 15, the user may speak a search word to themobile terminal 500. In detail, the user may speak the search word whilemaking a voice call through the mobile terminal 500.

The mobile terminal 500 may recognize the voice of the user through theinput unit 520. In detail, the input unit 520 of the mobile terminal 500may recognize the voice signal for what the user speaks.

The control unit 580 of the mobile terminal 500 may perform a controloperation to transmit the recognized voice signal to the display device100. In detail, the control unit 580 of the mobile terminal 500 maytransmit the recognized voice signal to the display device 100 while thedisplay device 100 operates in the search mode. In detail, the controlunit 580 of the mobile terminal 500 may transmit the recognized voicesignal to the display device 100 while the specific button is selected.

The display device 100 may receive the voice signal from the mobileterminal 500. The control unit 170 of the display device 100 may performa control operation to display a voice recognition window 1500 asillustrated in FIG. 15 by receiving the voice signal. The voicerecognition window 1500 is to indicate that the display device 100 isreceiving the voice signal from the mobile terminal 500.

The following description will be made with reference to FIG. 6 again.

The control unit 170 of the display device 100 may perform the searchbased on the received voice signal (S125).

As mentioned above, the control unit 170 of the display device 100 mayreceive a voice signal from the mobile terminal 500. The control unit170 may obtain a search word by analyzing the received voice signal. Thecontrol unit 170 may perform a search based on the obtained search word.For example, the control unit 170 may perform a web search, a mapsearch, an EPG search, an address book search in the mobile terminal100, or a photo search in the mobile terminal 100 based on the obtainedsearch word.

The control unit 170 of the display device 100 may control the displayunit 180 to display the searched result (S127).

The display unit 180 of the display device 100 may display the searchedresult.

In search a manner, the user may search for specific information whilemaking a voice call.

Hereinafter, a method that the display device 100 according to anembodiment of the present invention receives the voice signal from themobile terminal 500 and performs the search will be described withreference to FIGS. 16 to 29.

FIGS. 16 to 22 are views illustrating a method that the display deviceaccording to the first embodiment of the present invention receives avoice signal and displays the searched result.

As mentioned above, the control unit 170 of the display device 100 mayanalyze the voice signal from received the mobile terminal 500. Thecontrol unit 170 may obtain the search word based on the analysis resultof the voice signal.

The following description will be made regarding a method of obtainingthe search word according to an embodiment of the present invention. Forexample, the control unit 170 may analyze the voice signal and obtain aword matching the voice signal as the search word. Alternatively, thecontrol unit 170 may obtain at least one word having pronunciationclosest to that of the voice signal, as the search word.

In addition, the control unit 170 may analyze the voice signal and mayobtain only a noun included in the received voice signal as the searchword. In other words, the control unit 170 may obtain words, from whichall postposition particles are excluded, as search words.

In addition, the control unit 170 may obtain at least one noun includedin the voice signal and may obtain the obtained noun and a word obtainedby translating the noun in a different language, as the search word.

The control unit 170 may perform the search based on the obtained searchword through the above-described method.

The control unit 170 may perform a control operation such that thesearched result is displayed on the display unit 180 as illustrated inFIG. 16.

The display unit 180 may display a search result window 1600 togetherwith the video which is being viewed. The search result window 1600 mayinclude at least one of a total search result item 1601, a related videoresult item 1602, a recommended channel item 1603, a recommended relatedsearch word item 1604, or a speech recognition result item 1605.

The total search result item 1601 is an item for displaying, asthumbnails, results obtained by searching for a search word on a webbrowser.

The related video result item 1602 is an item for displaying at leastone video related to the search word. For example, the related videoresult item 1602 may include a video having the largest view countsrelated to the search word and the latest video related to the searchword.

The recommended channel item 1603 may include a channel list related tothe search word. The channel list related to the search word may includechannels directly related to the search word. Alternatively, the channellist related to the search word may include a channel that broadcasts aprogram related to the search word.

The recommended related search term item 1604 may indicate at least oneword related to the search word. For example, the recommended relatedsearch word item 1604 may include words that are frequently searched byother users in relation to the search word.

The speech recognition result item 1605 may include the voice signalrecognized by the control unit 170 and a search word found based on thevoice signal.

Accordingly, the user may keep viewing the video without interruptionwhile recognizing the searched result. In addition, the user mayvariously identify desired information through the items.

In addition, the user may view a specific item of the items bymagnifying the specific item. In detail, the user may select thespecific item from the search result window 1600 using the remotecontrol device 200.

As illustrated in FIG. 17, the control unit 170 may receive a commandfor selecting any one item from the search result window 1600. Thecontrol unit 170 may display content corresponding to the selected itemon the remaining area except for the search result window 1600. In otherwords, the control unit 170 may control the display unit 180 to displaythe content corresponding to the selected item on an area having thevideo displayed.

Alternatively, as illustrated in FIG. 18, the control unit 170 maydisplay the content corresponding to the selected item on the entirescreen. In detail, the control unit 170 may receive a command forselecting any one item from the search result window 1600. Accordingly,the control unit 170 may control the display unit 180 to display thecontent corresponding to the selected item on the area where the videoand the search result window 1600 are displayed.

As mentioned above, the user may select each item of the search resultwindow 1600 and obtain desired information in detail.

The display unit 180 may display at least one of a storage icon 1701, acaptured screen sharing icon 1702, or a link sharing icon 1703 whiledisplaying the searched result.

First, the storage icon 1701 will be described with reference to FIG.19. The storage icon 1701 is an icon for storing the searched result inthe display device 100 or the mobile terminal 500. The control unit 170may perform a control operation to store the searched result in thestorage unit 140 of the display device 100 or the memory 170 of themobile terminal 500 through the storage icon 1701. For example, thecontrol unit 170 may display a storage way menu 1900 when receiving acommand for selecting the storage icon 1701 through the pointer 205. Thestorage way menu 1900 may include an item to be stored in the displaydevice and an item to be stored in the mobile terminal. When receiving acommand for selecting any one item from the storage way menu 1900, thecontrol unit 170 may perform a control operation to store the searchedresult in the display device 100 or the mobile terminal 500 depending onthe command.

Next, the capture screen sharing icon 1702, and the link sharing icon1703 will be described with reference to FIGS. 20 to 22. The capturedscreen sharing icon 1702 is an icon for sharing a search result screen,which is currently displayed on the screen, with another user. Thesearch result screen may be transmitted to another user in the form ofan image through the captured screen sharing icon 1702. The link sharingicon 1703 is an icon for transmitting a URL address for viewing thesearched result, which is currently displayed on the screen, to anotheruser.

Accordingly, the user may select specific information through thecaptured screen sharing icon 1702 and transmit the selected informationto another user. Alternatively, the user may transmit a link addresssuch that another user may select and view specific information andrelated information through the link sharing icon 1703.

The following description will be made regarding the sharing way. Forexample, as illustrated in FIG. 20, the control unit 170 may receive acommand for selecting any one of the captured screen sharing icon 1702and the link sharing icon 1703 through the pointer 205. Accordingly, thecontrol unit 170 may display a sharing way menu 2000.

The sharing way menu 2000 may include an item of “send to the otherparty”, an item of “select from contacts and send”, and an item “sharein SNS”.

The control unit 170 may receive a command for selecting the item of“send to the other party” from the sharing way menu 2000. In this case,the control unit 170 may send a captured screen or a link address tocontacts of the other party being currently on a voice all through themobile terminal 500.

The control unit 170 may receive a command for selecting the item of“select from contacts”, from the sharing way menu 2000. In this case,the control unit 170 may control the display unit 180 to display acontact list 2100 stored in the mobile terminal 500 as illustrated inFIG. 21. The control unit 170 may receive contact data from the mobileterminal 500 and display the contact list 2100. The control unit 170 mayreceive a command for selecting any one of the contact list 2100 throughthe pointer 205. Accordingly, the control unit 170 may display a sendingwindow 2200 for sending a captured screen or a link address to anotheruser, as illustrated in FIG. 22. The control unit 170 may send thecaptured screen or the link address to another user through the sendingwindow 2200. In addition, the control unit 170 may send a messagethrough the sending window 2200, in addition to the captured screen orthe link address.

FIGS. 23 and 24 are views illustrating a method that the display deviceaccording to the second embodiment of the present invention receives avoice signal and displays the searched result.

The mobile terminal 500 may recognize a voice signal. As mentionedabove, the control unit 170 of the display device 100 may receive thevoice signal from the mobile terminal 500. The description of the methodof recognizing and receiving the voice signal is the same as the methoddescribed with reference to FIG. 15. In the example illustrated in FIG.23, the control unit 170 of the display device 100 may receive a voicesignal including “Restaurant D at Exit 1 in Station ABC”.

The control unit 170 may obtain a search word by analyzing the receivedvoice signal. The control unit 170 may perform a web search, a mapsearch, and a memory search in the mobile terminal 100 based on theobtained search word. For example, the control unit 170 may search theweb, the map, and the mobile terminal 500 for “Restaurant D at Exit 1 inStation ABC”.

The control unit 170 may display the searched result on the display unit180. In detail, as illustrated in FIG. 24, the controller 170 may splitthe display unit 180 into a plurality of areas, and display searchedresults in the split areas, respectively. For example, the control unit170 may split the display unit 180 into a first area 2401 to a fourtharea 2404. The control unit 170 may perform a control operation todisplay a video on the first area 2401, to display a map search resulton the second area 2402, may display a web search result on the thirdarea 2403, and to display a searched result from the mobile terminal 500on the fourth area 2404.

The number of split screens, the way to split the screen, and content (avideo or a searched result) to be displayed on each area may be set bydefault or may be arbitrarily set by a user.

FIGS. 25 and 26 are views illustrating a method that the display deviceaccording to the third embodiment of the present invention receives avoice signal and displays the searched result.

The mobile terminal 500 may recognize a voice signal. As mentionedabove, the control unit 170 of the display device 100 may receive thevoice signal from the mobile terminal 500. The description of the methodof recognizing and receiving the voice signal is the same as the methoddescribed with reference to FIG. 15. In the example illustrated in FIG.25, the control unit 170 of the display device 100 may receive a voicesignal including “How many gold medals did we win at the Rio Olympics?”.

The control unit 170 may obtain a search word by analyzing the receivedvoice signal. In detail, the control unit 170 may obtain at least oneword included in the voice signal, as the search word. The control unit170 may search for the at least one word, which is obtained,individually or the combinations of the at least one word. For example,when the control unit 170 receives a voice signal of “How many goldmedals did we win at the Rio Olympics?”, the control unit 170 may obtainwords of ‘Rio’, ‘Olympics’ and ‘gold medals’ as search words. Thecontrol unit 170 may search for each of words of ‘Rio’, ‘Olympics’ and‘gold medal’ individually. In addition, the control unit 170 may searchfor “Rio Olympics” and “Gold medals at the Rio Olympics”.

The control unit 170 may perform a control operation such that thesearch result screen is displayed on the display unit 180 as illustratedin FIG. 26. For example, the control unit 170 may split the display unit180 into a first area 2601 to a fourth area 2604. The control unit 170may display the video on the first area 2601, to display the searchedresult for ‘Rio’ on the second area 2602, to display the searched resultfor ‘Rio Olympics’ on the third area 2603, and to display the searchedresult for “Gold medals at the Rio Olympics” on the fourth area 2604. Inother words, the control unit 170 may display the video and searchedresults for the search words on the split areas, respectively.Alternatively, the control unit 170 may display only searched resultsfor the search words on the split areas. The searched result displayedon each area may be a searched result having the largest search countsin relation to the received voice signal.

FIGS. 27 to 29 are views illustrating a method that the display deviceaccording to the fourth embodiment of the present invention receives avoice signal and displays the searched result.

The control unit 570 of the mobile terminal 500 may recognize a voicesignal of the calling party together with a voice signal of the user.The mobile terminal 500 may send the voice signal of the user and thevoice signal of the calling party to the display device 100.

The control unit 170 of the mobile terminal 100 may receive the voicesignal of the calling party together with the voice signal of the user.The control unit 170 may control the display unit 180 to display thereceived voice signal on the display unit 180.

For example, as illustrated in FIG. 27, the control unit 170 may performa control operation to display a voice signal window 2700. The voicesignal window 2700 is a window for displaying the conversation betweenthe user and the calling party in the form of a caption. Accordingly,the control unit 170 may analyze the voice signal of the user and thevoice signal of the calling party to display a user voice 2701 andanother party's voice 2702 on the voice signal window 2700. The user mayvisually view the conversation between the user and the calling partythrough the voice signal window 2700.

In addition, the user may search for specific information through thevoice signal window 2700. In detail, as illustrated in FIG. 28, the usermay search for any one caption displayed on the voice signal window2700. The control unit 170 may receive a command for selecting any oneof at least one caption displayed on the voice signal window 2700through the pointer 205. The control unit 170 may obtain a search wordfrom the selected caption and perform a search. For example, the controlunit 170 may extract at least one noun from the selected caption andobtain the extracted noun as a search word.

As illustrated in FIG. 29, the control unit 170 may control the displayunit 180 to display a searched result. For example, the control unit 170may receive a command for selecting the caption of “What is the trafficsituation at XX town like?”. Accordingly, the control unit 170 mayobtain a word of ‘XX town’ or ‘traffic situation’ as a search word. Thecontrol unit 170 may display a searched result obtained by using theobtained word of ‘XX town’ or ‘traffic situation’ on the pop-up window2900. When the control unit 170 receives a command for selecting anothercaption in the state that the pop-up window 2900 is displayed, thecontrol unit 170 may display the searched result corresponding to thecommand on the pop-up window 2900.

The user has the effect of instantly searching for specific informationduring the conversation, while visually viewing the conversation withthe other party as mentioned above.

The display device 100 according to the present invention may displayresults searched by receiving the voice signal as described according tothe first embodiment to the fourth embodiment. However, this is providedonly for the illustrative purpose, and the display device 100 maydisplay the searched result by receiving the voice signal throughanother method.

The display device 100 may store the searched result in the storage unit140. The user may view the stored searched result thereafter.

Hereinafter, a method for storing and displaying the searched resultaccording to an embodiment of the present invention will be describedwith reference to FIGS. 30 to 32. FIGS. 30 to 32 are views illustratinga method that the display device stores and displays the searched resultafter terminating the call according to an embodiment of the presentinvention.

The control unit 170 of the display device 100 may perform a controloperation to automatically store a searched result after receiving avoice signal, performing a search, and terminating a call. Asillustrated in FIG. 30, the control unit 170 may store the searchedresult while displaying a storage notification window 3000. In addition,the control unit 170 may perform a control operation to display a windowfor asking a user whether to store the searched result. Alternatively,the control unit 170 may display a window for asking the user whether tostore the searched result in the display device 100 or the mobileterminal 500.

The control unit 170 may store the searched result in the storage unit140 by matching the searched result with the calling party. Accordingly,as illustrated in FIG. 31, the control unit 170 may display contacts ofthe other party and a search result icon 3101 corresponding to thecontacts, when displaying a contact list 3100.

The control unit 170 may receive a command for selecting the searchresult icon 3101 through the pointer 205. In this case, the control unit170 may display a window for the past searched results associated withthe contacts corresponding to the selected search result icon 3101 asillustrated in FIG. 32.

The user has an effect of checking the past searched results through thedisplayed search result window.

As mentioned above, the user may conveniently search for, store, andcheck specific information by using the display device 100 while makinga voice call.

According to an embodiment of the present invention, the above-describedmethod is implemented as code that can be written to aprocessor-readable recording medium and can thus be read by a processor.Examples of the processor-readable recording medium include a ROM, aRAM, a CD-ROM, a magnetic tape, a floppy disk and an optical datastorage.

The display device according to the present invention should not belimited to configurations and methods of the above-describedembodiments, and all or some of the embodiments may be selectivelycombined with one another to achieve various alterations.

What is claimed is:
 1. A display device comprising: a short-rangecommunication unit to receive a search request signal from a mobileterminal and receive a voice signal from the mobile terminal afterreceiving the search request signal; and a display unit to display aresult searched based on the voice signal.
 2. The display device ofclaim 1, further comprising: a storage unit to store the result searchedbased on the voice signal.
 3. The display device of claim 1, wherein theshort-range communication unit transmits the searched result to themobile terminal.
 4. The display device of claim 1, wherein the displayunit displays a sharing menu including an item for transmitting to acalling party and an item for transmitting to a selected person fromcontacts, and wherein the short-range communication unit transmits thesearched result to the mobile terminal of the calling party or a mobileterminal of the selected person from the contacts, based on selecteditem from the sharing menu.
 5. The display device of claim 1, furthercomprising: a control unit obtains at least one search word by analyzingthe voice signal, wherein the display unit displays the result searchedbased on the search word.
 6. The display device of claim 5, wherein thecontrol unit performs a web-browser search, a map search or a search onthe mobile terminal by using the search word.
 7. The display device ofclaim 4, wherein the control unit performs, when a plurality of searchwords are obtained, a search using each of the plurality of search wordsand a search using combination of the plurality of search words.
 8. Thedisplay device of claim 7, wherein the control unit splits a screendisplayed on the display unit into a plurality of areas and displays thesearched result on the plurality of areas that are split.
 9. The displaydevice of claim 6, wherein the control unit splits the screen displayedon the display unit into four areas and displays the video which isbeing viewed, a result of the web-browser search, a result of the mapsearch, and a result of the search on the mobile terminal on the splitareas.
 10. The display device of claim 1, wherein the display unitdisplays a user voice signal and a voice signal of a calling party ascaptions, and displays the result searched based on a selected captionwhen receiving a command for selecting any one of the captions.
 11. Thedisplay device of claim 2, wherein the storage unit automatically storesthe result searched when the call is terminated.
 12. The display deviceof claim 11, wherein the storage unit stores the searched result bymatching the result searched with a calling party.
 13. The displaydevice of claim 1, wherein the search request signal is a voice signalincluding a preset keyword.
 14. The display device of claim 13, whereinthe keyword is set by default or arbitrarily set through a user input.15. The display device of claim 1, further comprising: a control unitreceive the voice signal from the mobile terminal if being in connectedstate with the mobile terminal.
 16. The display device of claim 15,wherein the control unit turns on power of the display device anddisplays a message indicating a search is possible in response toreceiving the search request signal, when the power is turned off eventhrough being in the connected state with the mobile terminal.
 17. Thedisplay device of claim 15, wherein the short-range communication unittransmits a request signal for Bluetooth Low Energy (BLE) connection tothe mobile terminal if being not in the connected state with the mobileterminal.
 18. The display device of claim 17, wherein the control unittransmits the request signal for the BLE connection to the mobileterminal in response to receiving a command for selecting one buttonprovided in a remote control device.
 19. The display device of claim 1,further comprising: a camera; and a control unit to recognize that thesearch request signal has been received if the camera senses a userfacing a direction of the display device for a preset time.
 20. Thedisplay device of claim 1, wherein the display unit displays a screencorresponding to a search mode when the power of the display device ison and BLE connection to the mobile is on.